[5.4] Able to use multiple Redis cluster and non-cluster instances together #16696

alibo · 2016-12-07T21:33:13Z

This PR allows user to create and use multiple Redis cluster as well as non-cluster instances.

Sample configs:

Multiple Redis clusters:

    'redis' => [
        'clusters' => [
            'redis-cluster-1' => [
                [
                    'host'     => '127.0.0.1',
                    'port'     => 6379,
                    'database' => 0,
                ],
                [
                    'host'     => '127.0.0.1',
                    'port'     => 6380,
                    'database' => 0,
                ],
            ],
            'redis-cluster-2' => [
                [
                    'host'     => '127.0.0.1',
                    'port'     => 7379,
                    'database' => 0,
                ],
                [
                    'host'     => '127.0.0.1',
                    'port'     => 7380,
                    'database' => 0,
                ],
            ],
        ],
    ],

Multiple Redis clusters with different options:

    'redis' => [
        'clusters' => [
            'redis-cluster-1' => [
                'options' => [
                    'cluster' => 'redis',
                ],
                [
                    'host'     => '127.0.0.1',
                    'port'     => 6379,
                    'database' => 0,
                ],
                [
                    'host'     => '127.0.0.1',
                    'port'     => 6380,
                    'database' => 0,
                ],
            ],
            'redis-cluster-2' => [
                'options' => [
                    'cluster' => 'predis',
                ],
                [
                    'host'     => '127.0.0.1',
                    'port'     => 7379,
                    'database' => 0,
                ],
                [
                    'host'     => '127.0.0.1',
                    'port'     => 7380,
                    'database' => 0,
                ],
            ],
        ],
    ],

Single Redis cluster and two non-cluster instances:

    'redis' => [
        'default' => [
            'host' => '127.0.0.1',
            'port' => 16379,
            'database' => 0,
        ],
        'queue' => [
            'host' => '127.0.0.1',
            'port' => 26379,
            'database' => 0,
        ],
        'clusters' => [
            'redis-cluster' => [
                [
                    'host'     => '127.0.0.1',
                    'port'     => 6379,
                    'database' => 0,
                ],
                [
                    'host'     => '127.0.0.1',
                    'port'     => 6380,
                    'database' => 0,
                ],
            ],
        ],
    ],

It won't break anything! (unless you're using clusters as a key!)
(Maybe we can deprecate cluster option after merging this PR?)

   'redis' => [
        'cluster' => true,
        'default' => [
            'host' => '127.0.0.1',
            'port' => 16379,
            'database' => 0,
        ],
        'clusters' => [
            'redis-cluster-1' => [
                [
                    'host'     => '127.0.0.1',
                    'port'     => 6379,
                    'database' => 0,
                ]
            ],
            'redis-cluster-2' => [
                [
                    'host'     => '127.0.0.1',
                    'port'     => 7379,
                    'database' => 0,
                ]
            ],
        ],
    ],

With above config, we have 3 connections which all of them are clusters: default, redis-cluster-1, redis-cluster-2

I also add some tests for PhpRedis. They're not good but better than nothing!! If you can make them better, Please tell me or of course you can send an another PR :D

alibo · 2016-12-08T08:04:03Z

Yeah finally! I fixed the cs :))

I don't know why this command is not working, it's just freezing!

$ php-cs-fixer fix "src/Illuminate/Redis" --config-file=".php_cs"
Loaded config from ".php_cs".

GrahamCampbell · 2016-12-08T12:22:49Z

I don't know why this command is not working, it's just freezing!

I wouldn't recommend using php-cs-fixer locally. Just apply the diff StyleCI gives you, or, just don't bother. StyleCI will merge its own fixes after the PR is merged.

alibo · 2016-12-08T13:18:27Z

@GrahamCampbell Thank you :) I didn't know StyleCI also can fix the code style. 👍

GrahamCampbell · 2016-12-09T16:36:53Z

Please revert all the bad changes you've made to the phpdoc.

alibo · 2016-12-10T09:58:16Z

@GrahamCampbell phpstorm 😶
It's ok now :)

taylorotwell · 2016-12-12T15:03:51Z

Had someone raise another PR about Redis Cluster 3.0. Does this work with Redis Cluster 3.0?

alibo · 2016-12-12T20:36:49Z

@taylorotwell I didn't change those PhpRedis codes. With this PR, you just can connect to multiple Redis cluster instances, as well as, non-cluster instances at the same time. Before this, you can only have multiple non-cluster instances or just a single Redis cluster instance. ('cluster' => true|false)

But I've tested them and both Predis and RedisCluster work:

3 masters and 3 slaves (based on https://redis.io/topics/cluster-tutorial):
7000 7001 7002 7003 7004 7005

Configuring:

nano ~/redis-3.2.6/cluster-test/700{0..5}/redis.conf

port 7000 #7000-7005
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

Running redis servers:

cd ~/redis-3.2.6/cluster-test/7000 && ../../src/redis-server redis.conf
cd ~/redis-3.2.6/cluster-test/7001 && ../../src/redis-server redis.conf
cd ~/redis-3.2.6/cluster-test/7002 && ../../src/redis-server redis.conf
cd ~/redis-3.2.6/cluster-test/7003 && ../../src/redis-server redis.conf
cd ~/redis-3.2.6/cluster-test/7004 && ../../src/redis-server redis.conf
cd ~/redis-3.2.6/cluster-test/7005 && ../../src/redis-server redis.conf

Creating cluster:

./redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 \
 ◇ 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005

Cluster info:

Using 3 masters:
127.0.0.1:7000
127.0.0.1:7001
127.0.0.1:7002
Adding replica 127.0.0.1:7003 to 127.0.0.1:7000
Adding replica 127.0.0.1:7004 to 127.0.0.1:7001
Adding replica 127.0.0.1:7005 to 127.0.0.1:7002
M: 805950d657684787324c4865a4521079c648dee8 127.0.0.1:7000
   slots:0-5460 (5461 slots) master
M: cc311766ed77787dafb5dad6dca804a6d5ae774a 127.0.0.1:7001
   slots:5461-10922 (5462 slots) master
M: 5c2505e4f54c7ea1750300cab62287c2bb6c19b1 127.0.0.1:7002
   slots:10923-16383 (5461 slots) master
S: ca4364e034e089069ba01c09d14478e33c822eef 127.0.0.1:7003
   replicates 805950d657684787324c4865a4521079c648dee8
S: 5288746230933323e00c618979cf6e1ee11679d8 127.0.0.1:7004
   replicates cc311766ed77787dafb5dad6dca804a6d5ae774a
S: 86de908e51e7757431d6e42337024f69c60bebd7 127.0.0.1:7005
   replicates 5c2505e4f54c7ea1750300cab62287c2bb6c19b1

Laravel Redis Config (Predis):

   'redis' => [

        'client' => 'predis',

        'cluster' => false,

        'clusters' =>[ 
            'cluster-1' => [
                'options' => [
                    'cluster' => 'redis',
                ],
                [
                    'host' => '127.0.0.1',
                    'port' => 7000,
                ],
                [
                    'host' => '127.0.0.1',
                    'port' => 7001,
                ],
                [
                    'host' => '127.0.0.1',
                    'port' => 7002,
                ],
                [
                    'host' => '127.0.0.1',
                    'port' => 7003,
                ],
                [
                    'host' => '127.0.0.1',
                    'port' => 7004,
                ],
                [
                    'host' => '127.0.0.1',
                    'port' => 7005,
                ],
            ]
        ]

    ],

Laravel Redis Config (PhpRedis):

  'redis' => [

        'client' => 'phpredis',

        'cluster' => false,

        'clusters' =>[ 
            'cluster-1' => [
                [
                    'host' => '127.0.0.1',
                    'port' => 7000,
                ],
                [
                    'host' => '127.0.0.1',
                    'port' => 7001,
                ],
                [
                    'host' => '127.0.0.1',
                    'port' => 7002,
                ],
                [
                    'host' => '127.0.0.1',
                    'port' => 7003,
                ],
                [
                    'host' => '127.0.0.1',
                    'port' => 7004,
                ],
                [
                    'host' => '127.0.0.1',
                    'port' => 7005,
                ],
            ]
        ]

    ],

My php version:

PHP 7.1.0 (cli) (built: Dec  2 2016 03:30:24) ( NTS )
Copyright (c) 1997-2016 The PHP Group
Zend Engine v3.1.0-dev, Copyright (c) 1998-2016 Zend Technologies

Testing Predis cluster:

art tinker
Psy Shell v0.7.2 (PHP 7.1.0 — cli) by Justin Hileman
>>> $redis = app('redis');
=> Illuminate\Redis\PredisDatabase {#649}
>>> dump($redis->connection('cluster-1')->set('testing-1', 1));
Predis\Response\Status {#678
  -payload: "OK"
}
=> null
>>> dump($redis->connection('cluster-1')->set('testing-2', 2));
Predis\Response\Status {#678
  -payload: "OK"
}
=> null
>>> dump($redis->connection('cluster-1')->set('testing-3', 3));
Predis\Response\Status {#678
  -payload: "OK"
}
=> null

redis-cli -p 7000 -c
127.0.0.1:7000> get testing-1
-> Redirected to slot [12279] located at 127.0.0.1:7002
"1"
127.0.0.1:7002> get testing-2
-> Redirected to slot [8084] located at 127.0.0.1:7001
"2"
127.0.0.1:7001> get testing-3
-> Redirected to slot [4021] located at 127.0.0.1:7000
"3"

Testing PhpRedis cluster:

art tinker
Psy Shell v0.7.2 (PHP 7.1.0 — cli) by Justin Hileman
>>> $redis = app('redis');
=> Illuminate\Redis\PhpRedisDatabase {#649}
>>> dump($redis->connection('cluster-1')->set('testing-phpredis-1', 1));
true
=> null
>>> dump($redis->connection('cluster-1')->set('testing-phpredis-2', 2));
true
=> null
>>> dump($redis->connection('cluster-1')->set('testing-phpredis-3', 3));
true
=> null

redis-cli -p 7000 -c
127.0.0.1:7000> get testing-phpredis-1
-> Redirected to slot [15531] located at 127.0.0.1:7002
"1"
127.0.0.1:7002> get testing-phpredis-2
-> Redirected to slot [3272] located at 127.0.0.1:7000
"2"
127.0.0.1:7000> get testing-phpredis-3
-> Redirected to slot [7401] located at 127.0.0.1:7001
"3"
127.0.0.1:7001>

taylorotwell · 2016-12-12T21:02:30Z

So this is different from the current "cluster" option right? That option just did client side sharding which is not the true official "Redis Cluster", right? This now adds support for the actual official Redis Cluster stuff, correct?

taylorotwell · 2016-12-12T21:59:26Z

I've read through this and cleaned up a few things and got it merged. I am going to deprecate the "cluster" option in 5.4 and just use the clusters array introduced here instead.

alibo · 2016-12-12T22:22:34Z

That option just did client side sharding which is not the true official "Redis Cluster", right?

@taylorotwell Laravel already has true official "Redis Cluster". You should only add this to Redis config array:

'options' => [
    'cluster' => 'redis'
]

More info: https://github.com/nrk/predis#cluster and https://github.com/nrk/predis/wiki/Client-Options

This now adds support for the actual official Redis Cluster stuff, correct?

No, it allows you to use multiple Redis clusters. Also, you can use non-cluster instances for something else such as Redis geo, queue, etc. (Redis cluster has own limitations!)
Before this PR, if you set cluster to true, you only have one single cluster connection (default), without any non-cluster instances.

I am going to deprecate the "cluster" option in 5.4 and just use the clusters array introduced here instead.

Perfect 👌 Thanks :)

Able to use multiple redis clusters

471d41a

alibo changed the title ~~Able to use multiple Redis cluster and non-cluster instances together~~ [5.4] Able to use multiple Redis cluster and non-cluster instances together Dec 7, 2016

alibo added 3 commits December 8, 2016 11:24

fix cs and doc blocks

117de2a

fix cs

f516d6b

fix cs, please :(

95cc3f2

fix phpdoc

cd63104

taylorotwell merged commit cd63104 into laravel:master Dec 12, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[5.4] Able to use multiple Redis cluster and non-cluster instances together #16696

[5.4] Able to use multiple Redis cluster and non-cluster instances together #16696

alibo commented Dec 7, 2016

alibo commented Dec 8, 2016

GrahamCampbell commented Dec 8, 2016

alibo commented Dec 8, 2016

GrahamCampbell commented Dec 9, 2016

alibo commented Dec 10, 2016

taylorotwell commented Dec 12, 2016

alibo commented Dec 12, 2016 •

edited

Loading

taylorotwell commented Dec 12, 2016

taylorotwell commented Dec 12, 2016 •

edited

Loading

alibo commented Dec 12, 2016 •

edited

Loading

[5.4] Able to use multiple Redis cluster and non-cluster instances together #16696

[5.4] Able to use multiple Redis cluster and non-cluster instances together #16696

Conversation

alibo commented Dec 7, 2016

alibo commented Dec 8, 2016

GrahamCampbell commented Dec 8, 2016

alibo commented Dec 8, 2016

GrahamCampbell commented Dec 9, 2016

alibo commented Dec 10, 2016

taylorotwell commented Dec 12, 2016

alibo commented Dec 12, 2016 • edited Loading

Configuring:

Running redis servers:

Creating cluster:

Cluster info:

Laravel Redis Config (Predis):

Laravel Redis Config (PhpRedis):

Testing Predis cluster:

Testing PhpRedis cluster:

taylorotwell commented Dec 12, 2016

taylorotwell commented Dec 12, 2016 • edited Loading

alibo commented Dec 12, 2016 • edited Loading

alibo commented Dec 12, 2016 •

edited

Loading

taylorotwell commented Dec 12, 2016 •

edited

Loading

alibo commented Dec 12, 2016 •

edited

Loading